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Abstract. - The question of whether the Tsallis entropy is Lesche-stable is revisited. It is argued 
that when physical averages are computed with the escort probabilities, the correct application 
of the concept of Lesche-stability requires use of the escort probabilities. As a consequence, as 
shown here, the Tsallis entropy is unstable but the thermodynamic averages are stable. We further 
show that Lesche stability as well as thermodynamic stability can be obtained if the homogeneous 
entropy is used as the basis of the formulation of non-extensive thermodynamics. In this approach, 
the escort distribution arises naturally as a secondary structure. 



Introduction. — The concept of non-extensive ther- 
modynamics was introduced by Tsallis about 20 years 
ago [1] and has generated a large literature. The orig- 
inal idea was that for systems out of equilibrium where 
the Boltzmann distribution no longer holds, the Boltz- 
mann entropy could be replaced by a more general func- 
tion while maintaining the formalism of thermodynam- 
ics. In particular, maximization of the entropy under the 
usual constraints (normalized probabilities, fixed internal 
energy) yields the so-called q-Gaussian distribution that 
generalizes the usual Boltzmann distribution of classical 
statistical mechanics. While this seems straightforward 
now, it did in fact require considerable effort to arrive at 
the now-accepted form of the theory. A particular issue 
that was historically important and that remains prob- 
lematic is the notion of thermodynamic stability since the 
Tsallis entropy gives negative specific heats in certain cir- 
cumstances [2-5]. Nevertheless, this was one of the issues 
that motivated the advocation of nontrivial averaging pro- 
cedures in non-extensive thermodynamics [3]. 

A new controversy has arisen based on a recent paper 
by Abe where it is shown that averages computed within 
the non-extensive formalism are unstable in the sense that 
a small change in the distribution function can lead to a 
large change in the computed average [6] . This surprising 
result should be understood in a broader context wherein 
it was originally asked whether the Tsallis entropy is sta- 
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ble with respect to changes in the distribution. This was 
shown to be true by Abe [7] so the result that averages 
of observables are unstable while the entropy is stable ap- 
pears quite surprising. In this paper we question whether 
either fact has actually been proven. In short, our ar- 
gument is that Lesche-stability is motivated by making 
correspondence with an experimental procedure and that 
this means it should be understood in terms of the proba- 
bilities that govern the observation of a given microstate. 
In the usual formulation of non-extensive thermodynam- 
ics, those are the escort probabilities. When understood 
in this way, it is easy to show that the Tsallis entropy is 
not Lesche-stable. 

This would appear to create an uncomfortable situation 
in which the Tsallis formulation is not Lesche-stable and 
in which thermodynamic stability is also problematic. We 
contend that this can be resolved by a shift of viewpoint in 
which the physical probabilities are taken as being funda- 
mental. While the Tsallis entropy cannot be satisfactorily 
formulated in this way [8], a closely related functional, 
the homogeneous entropy, appears as a natural alterna- 
tive. We show that the homogeneous entropy is in fact 
Lesche stable, gives positive-definite specific heats, yields 
the usual q-Gaussian distributions when maximized and 
gives rise to a consistent thermodynamics. 

Non-extensive thermodynamics. 

Tsallis formalism. The usual non-extensive formalism 
can be illustrated as follows. Consider a system composed 
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of some number, n, of microstates and let pi be the prob- 
ability associated with the i-th microstate. The Tsallis 
entropy is computed as 



S q = 



(i) 



where q is the index characterizing the entropy functional. 
The limit q — > 1 gives the usual Boltzmann entropy. One 
subtlety in the theory of non-extensive thermodynamics is 
that the average of an observable, O, that takes the value 
Oi in the i-th state is evaluated using the so-called escort 
distribution [9] giving 



<0> = E 



pi 



=i 2^j=iP 3 



(2) 



Let the energy observable be U and let it take on the value 
gj in the i-th state. Then maximization of the entropy 
under the constraints of fixed average energy, (U) = U, 
and of normalization, 1 = Y^i=iPi-> gives the q— Gaussian 
distribution, 



Pi 



(l-(l-q)Zf^(e 3 -U))^ 



(3) 



with 



N 



z « = E (!-(!- «) z r'p & - u ))+ 1 • ( 4 ) 

i=i 

(Here, the notation {x) + means x if x > and zero oth- 
erwise.) It is straightforward to show that if the energy 
levels are functions of some parameter, ei = Si (A), then 



dU_ B -idS_ dW 



(5) 



where the work is defined as dW = Y^i=\Pi^ e i- This is 
recognized as the first law of thermodynamics. 

The peculiar nature of the average, based as it is on the 
escort distribution, naturally suggests a reformulation of 
the theory If one defines the new quantity, 



which is invcrtiblc, 



Pi = 



P% = 



P- 
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then the Tsallis entropy becomes 

~ i-(Er=i^ 1/? )" 



q-l 

and averages are computed as normal, 



(8) 



Extremizing this entropy subject to the usual constraints 
again gives a q-Gaussian distribution, but with exponent 
q/q—1 rather than q. In the following, we refer to the orig- 
inal, more familiar formulation of non-extensive thermo- 
dynamics in terms of the Tsallis entropy as the "little-p" 
picture and the reformulation given here as the "big-P" 
picture. All of this has long been known in the litera- 
ture of non-extensive thermodynamics [10]. The little-p 
formulation is generally favored because of one important 
difference: the Tsallis entropy is a concave function of the 
probabilities whereas the same is not true of the big-P 
entropy. (Concavity is assumed to be required of a gen- 
eralized entropy even though the connection between con- 
cavity and thermodynamic stability is complicated by the 
non-additivity of the entropy [11].) 

On the other hand, there is another important differ- 
ence between the two pictures from a more physical point 
of view: the escort probabilities have the interpretation 
of being a measure of the likelihood of finding the system 
in a given microstate. This is obvious if one considers an 
ensemble of systems in which case the fact that the aver- 
ages are computed via Eq. (9) implies that the fraction 
of systems in microstate i must be Pj. This is to say that 
the usual ensemble interpretation of statistical averages 
implies a frequentist interpretation of the escort proba- 
bilities. Conversely no such ontology can be imposed on 
the small-p probabilities. When using the small-p formu- 
lation to express the average (O) = Y^i=i — r^«i ^ 
is clear that pi does not appear to measure the frequency 
of anything. Furthermore, experiments, which measure 
averages, are always going to be determining the escort 
(big-P) distribution and not the small-p distribution. The 
fact that both are q-Gaussians means that this distinction 
is not of practical importance, but the distinction is real. 
The question then arises, is this distinction ever of practi- 
cal importance? We next show that it plays a critical role 
in the discussion of stability of the entropy functional. 

Stability. — The discussion of stability is based on 
ideas first introduced by Lesche while studying the Renyi 
entropy [12]. Leschc-stability is defined in terms of 
countable families of probability distributions, W n = 

(w^\...,Wn^. An entropy function, (W n ) is said 
to be Lesche-stable if given two probability distributions, 
W n and W n — (w^', ...,wi™^, for all e > 0, there exists 
S > such that 



5>\W n -W^\ = Y, 



w 



{n) 



- W 



(n)t 



implies that 



e > 



S {n) {W n ) ~ (W n ) 



(10) 



(11) 



»=i 



(n) 

^ where Sma X is the maximum value possible for the entropy 
functional. The quantity \W n - W' n \ in Eq.(10) is the L\ 
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norm, and the implication (10) — > (11) is required to hold 
with fixed e,S in the limit of n — > oo. This definition is 
technical, but the idea is simple: when the the probabili- 
ties change, the change in the entropy should be small if 
the change in the probabilities is small. By formulating 
the definition in terms of a limit, it covers both the case 
of probability assignments with finite support (i.e. only 
some fixed number of elements are nonzero) as well as as- 
signments with infinite support. Lesche proved that the 
Boltzmann entropy is stable, but gave a simple example 
showing that the Renyi entropy is not stable [12]. Abe 
has given a proof that the Tsallis entropy in the small-p 
picture is stable [7]. 

One question that arises immediately is why use the 
L\ norm in measuring the distance between two distri- 
butions? In his original paper [12], Lesche stated that 
this measure can be related to the difficulty in experimen- 
tally distinguishing two probability assignments. The full 
argument was given recently by Abe, Lesche and Mund 
[13] and goes as follows. To tell whether W or W is 
the correct description of a given ensemble of systems, 
one performs some number, Z, of experiments which, in 
each instance, determine which microstate the system is 
in. Of course, microstates are usually not directly ob- 
servable, but for example, one might measure the energy 
and from this deduce the microstate (or a set of compat- 
ible microstates). The result is a sequence of observed 
microstates, S = (si,...,sz)- If the correct distribu- 
tion is W,the probability to observe this sequence is the 
product of the probabilities of observing each microstate, 



Tsallis entropy using the metric 



P(s; W) = w Sl w s , 



Abe et al. define the test of 



whether or not the distribution is W by requiring that, 
prior to the experiment, the experimenter specify a col- 
lection of possible results, C = jS*^}. The probability 
that the observed S € C is a = J2j P (Sji W), while the 
probability (3 = 1 — . P (Sj; W) is the probability that, 
if the true distribution is W' , the observation will not be 
in the set C. In order to be an effective test, the set C 
must be constructed so that both a and (3 are large: i.e., 
so that if W is correct, the observation is likely to be in 
C and if W is correct, the observation is likely to not be 
in C. They show that for given Z, no such test is possible 
if the L\ difference between W and W is too small. 

The key observation here is that the relevance of the 
L\ metric is based on an estimation of probabilities of 
observing certain outcomes, that is on the basis of a frc- 
quentist measure. Therefore, within the framework of 
non-extensive thermodynamics, these probabilities neces- 
sarily correspond to the escort probabilities. Thus, to test 
Lesche-stability in this formalism, the L\ metric must be 
applied to the escort probabilities. Demonstrations based 
on separation of the small-p distributions measured by 
the L\ metric do not correspond to the concept of Lesche- 
stability. An alternative statement, entirely within the 
little-p picture, is that one must check stability of the 
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The Tsallis entropy is unstable. So, is the Tsallis en- 
tropy Lesche-stable? The answer is that it is not. Con- 
sider the case q < 1 and let P be the uniform distribution, 



Pi 
Pi 



and 



— for all i, and P' be given by P{ 



£ + 



and 



n 2(n-l) 



for n > i > 1. Then, it is obvious that 



\p-p'\ Li = s 



S g (P) = 
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(13) 



(14) 



This happens to be equal to S q ^ max . The perturbed dis- 
tribution gives 

i- ((^i) i/f +(»-D ft- ^n - * 

s q (P') = — ^ 4^ 



q-l 



Hence, 



lim 



S g (P) 



S q (P') 



-'t/.max 



lim 



(!)" 



(16) 

So, no matter how close the distributions (i.e. no matter 
how small S) , the difference between the entropies is finite. 
There is no difference if the calculation is translated into 
the small-p picture. 

The homogeneous entropy. — To summarize, our 
arguments show that the Tsallis entropy with linear av- 
erages is Lesche-stable (as proven by Abe) but with the 
escort distribution it is Lesche-unstable. On the other 
hand, it is known that both formalisms give negative spe- 
cific heats giving rise to questions of thermodynamic sta- 
bility [2,4, 11]. In fact, this was one issue that led to the 
search for an alternative to linear averages [3]. So, is there 
a non-extensive formalism that is both Lesche-stable and 
that gives positive specific heats? Given the intimate con- 
nection between the concept of Lesche-stability and the 
physical probabilities, one might wonder if it would make 
more sense to use the big-P entropy, Eq.(8) together with 
the linear averages as a starting point. As it happens, 
this is unsatisfactory because of the fact that the big-P 
entropy functional is not concave [8] . There is, however, a 
closely related function known in the information theory 
literature as the homogeneous entropy [14] given by 



i 



(17) 
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which is concave for all positive q, is maximized by the \Si\ < S < 1, one has |<5 4 | 1/q < \5i\ so 
uniform distribution and is extensible [15]. Our proposal is 

that this be used in conjunction with the linear averaging / / ™ 1 ,\ q ( ™ \ q 

procedure as a basis for the formulation of non-extensive S (P ) < I Pi I + I |^»| I = S (P) + S q . (24) 

thermodynamics. When maximized under constraint of \*=i / \»=i / 

normalization and fixed internal energy, the result is a q- Lesche _ stab i lity follows immc diatcly. For q > 1 the proof 

Gaussian, is gjjghtly more complicated. Note that if x > y > 

_ 1 . l _ q \\Th anc ^ ^ > t ^ ien 5 ( x ~ v) xq 1 — x<1 ~ y 9 ' w hich follows 

P i =Z q { 1 -( 1 - , l) Z q P( £ j- U )) ( 18 ) from the fact that q(x — y) x q ~ x — x q + y q is monotoni- 

cally decreasing as a function of y. Making all the same 
assumptions as above, this implies that 

(n \ ~T=i 

(M> (±*»)'-(±#')' <»> 

- £( 1 -p-,) g <":, p n* 

i=1 V Z\ ) Next, note that for <? > 1 and x > y > 0, x 1 / 9 - y 1 ^ < 

™ ,., rjl . rn n , , (x — y) 1 ^ 9 as follows from the fact that (x — y) 1 ^ — x x l q + 

I he equality ol the second and third lines follows from the i /„ . r ,. r mi 

. . . y is convex as a lunction ol y. thus 
constraint on tnc average energy, irom tins expression, it 

is straightforward to show that / n \i / n \i 

(e^ /, )-(e^ , j ( 26 ) 

— f=/3 (20) Vi=i / \i=i ) 

so that the Lagrange multiplier /3 corresponds to the in- < 9 I |^i| 1//9 J I J]] Pj ^ 9 I 

verse of the thermodynamic temperature. Furthermore, / \»=i / 

the specific heat is .i-i.ii.jii-- 

lhe two sums on the right can be bounded by maximiz- 
ed - (i- 1 ) 2 2 2q-i i n g ^™ =1 x^ 9 subject to J2i=i x i — 7 using a Lagrange 
Cv = df = qZq " E ^ ^ ~ ^ P i 9 ( 21 ) multiplier. The result is that x t = j/n and the sum is 



1/9 \ 



1 / n \ 1 



7 l/g n (g-l)/g glvmg 

which is positive definite. It is also easy to show that if 

the energies are a function of some external parameter, / ^ p 'i/g \ / p i/g 1 / 97 \ 

A, then I 2^ ^ I - I 2^ ^ I ^ 

9C/ as* - ae Wl 7 Vi=1 ' 

dx=P ^T + E^af ( 22 ) < ^ („(«-D/fl) 9 = ^/g n g-i 

which confirms the second law of thermodynamics for this wn ich, after normalization, implies Lesche-stability. 

m °d e l- Finally, we can make contact with the Tsallis entropy 

Lesche-stability is also easy to show. Let Si = P { — Pi, as follows. The normalization condition can be enforced 

E \Pi ~ p * I = ^ < 1 and assume without loss of generality b eliminating one of the degrees of freedom, i.e. by using 
that S (P') > S (P) where S (P) = (q-l) (Sf (P) + l) . 
Then, for q < 1, one has that 



q 



P n = 1 - p i (28) 



s(p') = E(^+^) 1/9 ( 23 ) 



However, this is awkward as it treats one degree of freedom 

( n 1/ \ 9 differently from the others. A more symmetrical way to 

E \Pi + $i\ q J impose it would be to introduce auxiliary quantities, Ui, 

i=1 ' and to write 

P = ^- (29) 
i u j 



This is completely general. Note that it is degenerate as 
The last line follows from the Minkowski inequality (a gen- {ui} and {Xui} give the same {pi}- Since pi > 0, all of 
eralization of the triangle inequality) [16]. For q < 1 and the ttj must be either positive or negative. Again, without 
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loss of generality, we can take them all to be positive. In 
terms of these, the entropy becomes 

s>) jE..^y(^ ,)-'-> (30) 

We can simplify this by using the freedom mentioned 
above to impose the single constraint, 



1/9 



(31) 



1=1 



which is equivalent to choosing a particular scaling of the 
u's. Note that since all the > 0, this implies that they 
are all also less than one. Then, it appears to be more 
convenient to introduce Vi 



u X J q e [0, 1] giving 



Pi = 



1 T n v q 



(32) 
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The "escort probabilities" therefore arise naturally as a 
way of encoding the normalization constraint. However, it 
is clear in this interpretation that the {vt} are not physical 
probabilities but just quantities that happen to be positive 
and to sum to one. The form of the homogeneous entropy 
written in terms of the {vi} is known in the literature and 
is called the "normalized Tsallis entropy" [17]. However, 
note that the normalized Tsallis entropy is only concave 
for q < 1 so the homogeneous entropy is a more general 
starting point. 

Concluding Comments. — Our conclusion is that 
when the concept of Lesche stability is properly applied 
within the usual formalism of non-extensive thermody- 
namics, the Tsallis entropy is just as unstable as the Renyi 
entropy originally considered by Lesche. Whether or not 
this conclusion, based on particular admittedly artificial 
examples, is physically relevant and would have practical 
implications is a question of on-going debate [18] just as 
was the question of the stability of the Renyi entropy [19]. 
However, arguments such as those given in [18] should be 
reconsidered in light of the correct application of the L\ 
measure. 

A further consequence which follows from the big-P 
formulation concerns the stability of the averages of ob- 
servables. In a recent demonstration they were shown to 
be unstable in the non-extensive formalism [6]. This re- 
sult was obtained using the small-p picture where the in- 
terpretation of the probabilities is problematic, while it 



was shown that in the classical formulation of statistical 
mechanics with linear averages, stability was guaranteed. 
This is precisely in accordance with our arguments: since 
the L\ measure should be applied to the escort probabil- 
ities (i.e. the big P's), it follows that the averages are 
stable. 

In summary, the use of the L\ norm in framing the con- 
cept of Lesche-stability is justified by considering an ex- 
perimental test designed to distinguish different hypoth- 
esized probability distributions. As such, it is a prop- 
erty not only of the form of the entropy but also of the 
means used to relate the "probabilities" occuring in the 
entropy to experiment. Our conclusion is that the result 
of Abe [7] should be interpreted as demonstrating that 
the combination of Tsallis entropy and linear averages is 
stable while the combination of Tsallis entropy with the 
escort distribution averages is unstable (as shown by our 
example above). This being the case, the further obser- 
vations concerning the stability of the averages [6] only 
reinforces these conclusions. 

On the other hand, we have shown that the homoge- 
neous entropy with the usual linear averaging procedure 
provides a satisfactory starting point for the development 
of non-extensive thermodynamics. We note that this is 
not the first time the homogeneous entropy has occured 
in the context of non-extensive thermodynamics. For ex- 
ample, Lavenda and Dunning-Davies have used it as an 
example illustrating that certain features of the Tsallis 
entropy are not unique [20] . In the form of the normalized 
Tsallis distribution, it appears to have first been discussed 
by Rajagopal and Abe [17] where it was noted that it is 
only concave for q G [0,1]. Abe subsequently concluded 
that it is not Lesche-stable, but by our interpretation, this 
argument shows instability when the linear averaging pro- 
cedure is used [7]. We also note that Lenzi et al had al- 
ready demonstrated that the normalized Tsallis entropy 
gives positive specific heats [21]. Our main contribution 
has been to note that the homogeneous entropy is more 
general than the normalized Tsallis entropy (because it is 
concave for all values of q) and to show it is Lesche-stable. 
Together with other properties, such as the positivity of 
the specific heats, we suggest this makes it a preferred 
starting point for the development of non-extensive ther- 
modynamics. 

Finally, this approach also sheds light on the underly- 
ing ontology of the formalism. The concept of the es- 
cort distribution has become an accepted part of the non- 
extensive formalism but, as we have discussed, it is hard 
to understand in what sense the small-p quantities are 
"probabilities" as opposed to a set of quantities that hap- 
pen to be positive and normalized. Nevertheless, vari- 
ous arguments have also been given for using the escort- 
distribution rather than the linear averaging procedure 
[22,23]. In our formulation, this ambiguity is clarified: 
the only probabilities are the physical probabilities used 
to compute averages. The escort distribution arises as a 
natural way to simplify the form of the entropy, but the 
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equivalent of the small-p variables are clearly quantities 
to which no physical significance attaches. 
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